Classroom model, model classroom: computer-supported methodology for investigating collaborative-learning pedagogy

We have been exploring the potential of agent-based modeling methodology for socialscience research and, specifically, for illuminating theoretical complementarities of cognitive and socio-constructivist conceptualizations of learning (e.g., Abrahamson & Wilensky, 2005a). The current study advances our research by applying our methodology to pedagogy research: we investigate individual and social factors underlying outcomes of implementing collaborativeinquiry classroom practice. Using bifocal modeling (Blikstein & Wilensky, 2006a), we juxtapose agent-based simulations of collaborative problem solving with real-classroom data of students’ collaboration in a demographically diverse middle-school mathematics classroom (Abrahamson & Wilensky, 2005b). We validate the computer model by comparing outcomes from running the simulation with outcomes of the real intervention. Findings are that collaboration pedagogy emphasizing group performance may forsake individual learning, because stable division-of-labor patterns emerge due to utilitarian preference of short-term production over long-term learning (Axelrod, 1997). The study may inform professional development and pedagogical policy (see interactive applet: http://ccl.northwestern.edu/research/conferences/CSCL2007/CSCL2007.html). Background and Objective We present a new methodology for developing and critiquing education theory, agent-based modeling. Agent-based modeling (hence ABM) has been increasingly used by natural scientists to study a wide range of phenomena such as the interactions of species in an ecosystem, the interactions of molecules in a chemical reaction, the percolation of oil through a substrate, and the food-gathering behavior of insects (e.g., Bonabeau, Dorigo, & Théraulaz. 1999; Wilensky & Reisman, 1998, 2006). Such phenomena, in which the elements within the system (molecules, or ants) have multiple behaviors and a large number of interaction patterns, have been termed complex and are collectively studied in a relatively young interdisciplinary field called complex systems or complexity studies (e.g., Holland, 1995). Typical of complex phenomena is that they lend themselves to two or more layers of description—e.g., collisions of particles in a gas chamber are the “micro” events, and pressure is the “macro” event— and the cumulative (‘aggregate’) patterns or behaviors at the macro level are not premeditated or directly actuated by any of the “lower-level” micro elements. For example, flocking birds do not intend to construct an arrow-shaped structure. Rather, each element (“agent”) follows its “local” rules, and the overall pattern arises as epiphenomenal to these multiple local behaviors—the overall pattern emerges. Specialized computer-based environments (Collier & Sallach, 2001; Langton & Burkhardt, 1997; Wilensky, 1999) have been developed as research tools for investigating complex phenomena (North et al., 2002; Wilensky, 2001; Wilensky & Reisman, 2006). The agents can be instantiated in the form of a computer program that specifies their rule-based behaviors. ABM is thus particularly powerful for studying complex phenomena, because once the modeler assigns agents their local rules, the modeler can set these virtual agents into motion and watch for any overall patterns that arise from the agents’ interactions. E.g., the modeler might assign a group of virtual birds a set of rules and then watch their interactions to see whether typical flock structures emerge (Reynolds, 1987). Whereas initially complex-systems methods and perspectives arose from the natural sciences, complexity, emergence, and microand macro levels of description of phenomena are all highly relevant to research in the social sciences. Indeed, the recent decades have seen a surge in social-science studies employing ABM (Axelrod, 1997; Diermeier, 2000; Epstein & Axtell, 1996). Learning, too, we argue, can be construed as a complex phenomenon, and thus ABM is a potentially powerful research tool conducive to the investigation of patterns, including structures and rules, underlying the emergence of learning. Specifically, we are proposing to use ABM in investigating the social dynamics underlying participation patterns observed when students interact around collaborative classroom assignments, such as construction projects. Thus, whereas our paper deals squarely with ‘mice, minds, and Abrahamson, D., Blikstein, P., & Wilensky, U.: (2007) C. Chinn, G. Erkens, & S. Puntambekar (Eds.), Proceedings of the Computer Supported Collaborative Learning (CSCL) Conference (Vol. 8, Part 1, pp. 46 – 55). NJ: Rutgers University. CD-ROM. http://ccl.northwestern.edu/research/conferences/CSCL2007/CSCL2007.html Abrahamson, Blikstein, & Wilensky (2007) [email protected] societies’—the theme of the CSCL 2007 conference—we are not so much dealing with computer-supported collaborative learning as such, as much as with computer-supported inquiry into collaborative learning (CSiiCL). Nevertheless, we hope to lay out agenda and methodology to facilitate synergies between the CSCL and complexitystudies communities—synergies that increase understanding, within education research, of mechanisms and practice pertaining to individual learning within social contexts. Thus, armed with computers as methodological tools for ultimately improving collaborative learning, this paper is about computer-supported collaborative learning. We have been working with the NetLogo (Wilensky, 1999) multi-agent modeling-and-simulation environment. A vision of the NetLogo development effort is that building simulations will become common practice of natural/social-sciences scholars investigating complex phenomena: the scholars themselves—not hired programmers—build, run, and interpret the simulations (Tisue & Wilensky, 2004; Wilensky, 2003). The new lenses of ABM, we believe, will enable education researchers to explore, articulate, develop, and share an intuition we have struggled to study rigorously and express coherently: the intuition that individuals and communities are interdependent through myriad dynamic reciprocities (Cole & Wertsch, 1996; Greeno, 1998). In the remaining sections of this paper, we: (1) introduce the case study—a collaborative construction project in a demographically diverse middle-school mathematics classroom studying combinatorial analysis (Abrahamson, Janusz, & Wilensky, 2006; Abrahamson & Wilensky, 2005b); (2) discuss the rationale, design, and implementation of a complexity-based analysis of the case-study’s participation patterns, i.e., an agent-based model that purports to simulate this phenomenon; (3) introduce ‘bifocal modeling’ (Blikstein, Abrahamson, & Wilensky, 2006), a computer-assisted research technique for juxtaposing real and simulated data toward calibrating the simulation such that it emulates the real data—we demonstrate this juxtaposition by aligning participation patterns in our classroom data with simulated patterns emerging in the ABM; (4) report findings; and (5) offer concluding remarks on the implications of this study and the limitations of ABM and suggest directions for further research. Case Study: Emergence of a Stratified Learning Zone in a Collaborative Project in a Demographically Diverse Mathematics Classroom Complexity-studies methodology is particularly suitable for understanding student learning in pedagogical frameworks that support individual agency. When such classrooms engage in collaborative construction projects, participation patterns emerge, some that may be undesirable, from the educator’s perspective. As we explain, below, these patterns emerge through iterative student-to-student negotiation of roles vis-à-vis students’ skills and their interpretation of the overall classroom objectives. When these objectives are taken to be production rather than learning, inequitable participation patterns may emerge, because students are rewarded for their contribution to production rather than for their learning. The interactions of these two reward systems (the first, indexing students’ contribution toward successful completion of a group project; the second, indexing students’ own learning) is a complex system—ABM enables us to study the nature of students’ iterative negotiations that give rise to the inequitable participation patterns. Thus, simulating participation patterns could provide designers and teachers valuable tools for running equitable classrooms. In particular, understanding the emergence of inequitable participation may help educators formulate responses that temper production to the benefit of learning. We now explain the case study that we investigated using ABM. The Combinations tower: A Combinatorial-Analysis Collaborative Project The current investigation uses data from a design-based research study of middle-school students’ mathematical cognition pertaining to the topic of combinatorial analysis (Abrahamson, Janusz, & Wilensky, 2006; Abrahamson & Wilensky, 2005a). Central to the study was an implementation of a challenging classroom collaborative project—the construction of the combinations tower, the exhaustive sample space of a 3-by-3 grid of nine squares that can each be either green or blue (for a total of 512 distinct “9-blocks”). The classroom, working in groups, created all the 9-blocks and assembled them into a very tall “histogram.” This histogram consisted of 10 columns running from “no-green” through “9-green” (the columns’ heights were, respectively, 1, 9, 36, 84, 126, 126, 84, 36, 9, 1—coefficients in the binomial function [a + b]). The Stratified Learning Zone: Group Dynamics From an Emergence Perspective Data analysis revealed unanticipated participation patterns. Namely, individual students operating within groups assumed by-and-large restricted roles that we named: (a) “number crunchers”; (b) designers; (c) producers; (d) implementers; (e) checkers; and (f) assemblers; and in addition, some students operated between groups as (g) Abrahamson, Blikstein, & Wilensky (2007) [email protected] ambassadors. We demonstrated the descending mathematical challenge of the a-through-f roles, e.g., the designers initiate combinatorial-analysis strategies, the implementers carry out these strategies, and the assemblers glue the 9blocks onto a poster. We demonstrated that students’ individual roles were related both to their mathematical achievement, as reported by the teacher, and their demographics. We argued that these roles were emergent and that they affected the students’ learning opportunities and self image and that therefore it is important to understand how some students landed up on the lower rungs of the production line—how a stratified learning zone emerged. A stratified learning zone is a design-engendered hierarchy of students’ potential learning trajectories along problem-solving skill sets, each delimited in its conceptual scope, and all simultaneously occurring within a classroom. In comparison, the term continuous learning zone depicts a space wherein students can each embark from a core problem, sustain engagement in working on this problem, and build a set of skills wherein each accomplishment suggests, contextualizes, and supports the exploration and learning of the successive skill, so that a solution path is learned as a meaningful continuum. Validation Through Feedback From the Students and Teacher Based on interviews with the teacher and the students, we formulated the following agent-based explanation of the emergence of student task distribution. Students’ roles emerged as a function of individual student interactions: Within a group, once a student realized that he had reached his limit in terms of mathematical problem solving as compared to another student within that group, the first student would often capitulate to his group-mate the task of pursuing that mathematical problem, and then she would take over, relegating to him a necessary task that was within his zone of achievement, thus freeing herself to focus on the problem he had abandoned. A network of symbiotic relationships crystallized as the more advanced students assumed leadership of their groups and as the emergent task specifications were articulated in terms of student roles and student-to-student and group-to-group negotiated partnerships. The likelihood of an individual student dominating another was affected by personality traits: of the mathematically advanced students, those who were less socially fluent preferred to work individually, whereas “bossier” students were more likely to assign tasks to other group members. Exploration Vs. Exploitation: A Perennial Tradeoff of Collaborative Inquiry? When a classroom that is engaged in collaborative project-based activity progresses towards successful completion of the project, could there be any justification to tamper with this progress? And yet, is a facilitator ethically permitted to sacrifice individual students’ learning so as ensure the completion of the project? To address this design-and-facilitation dilemma, we will now turn to a complexity-studies perspective on organizations. One could arguably model the study classroom as an organization, a collective of individuals with some shared objective and a modus operandi for working towards this objective. There are no monetary stakes involved, but certain roles enable some students to gain knowledge capital, whereas other roles do not. Our motivation to model the classroom as an organization is that construction projects may tacitly import to the learning space ethics, ethos, and praxis of working spaces that may not be entirely beneficial for all students. Axelrod and M. D. Cohen (1999) discuss exploration versus exploitation, a tradeoff inherent in complex adaptive systems, such as organizations. For instance, in allocating resources, an organization must determine which strategy will maximize its benefits—“mutating” to check for better fits with the changing environment or stagnating and cashing in on a proven model of success. Typically, “the testing of new types comes at some expense to realizing benefits of those already available” (Axelrod & M. D. Cohen, 1999, p. 44). We submit that a classroom can be seen as a complex adaptive system (Hurford, 2004), at least in terms of students’ within-group free-range agency in problem solving and the interactions that shape these agencies. Initially, all students are explorative. Yet, once a functioning coordination scheme has evolved that is apparently well adapted to the environment, i.e. the classroomas-a-whole is apparently progressing along a trajectory towards successfully completing a prescribed task and positive sanctioning is received from the forces that be (the facilitators), an implicit quietus is set on any further exploration, and the group achieves dynamic stability. From that point on, the individual cogs in the production mechanism hone their skills and produce (see Durkheim, 1947, for a social critique of the division of labor). Finding: Some Answers, New Questions When students are given the freedom to explore a problem collaboratively, both remarkable and undesirable group behaviors may emerge. It is not a zero-sum game—these “pros and woes” need not cancel each other out. An experienced and able teacher who anticipates this emergence and is sensitive to unforeseen behavior can steer this sensitive dependence so as to optimize student sharing and learning opportunities. The proposed Abrahamson, Blikstein, & Wilensky (2007) [email protected] methodology introduced in this paper may provide education researchers, designers, and practitioners tools for understanding classroom dynamics such that they can identify points of leverage for working with students’ natural behavioral inclinations to achieve equitable participation. The next section explores this possibility. Implementing a Theoretical Model of the Case-Study Emergent Classroom Participation Pattern in the Form of “Runnable” Agent-Based Procedures In this section, we demonstrate the applicability of ABM methodology for the investigation of pedagogical practice by explaining our design rationale for simulating the emergence of a stratified learning zone in a virtual classroom. Also, we demonstrate the iterative nature of this methodology by describing some of our key understandings, along the modeling process, that informed the improvement of the model. Whereas this paper is primarily methodological—we use particular research content so as to demonstrate an investigation technique—the reader may disagree with our theoretical model of the causes of stratification. We welcome such disagreement, because we regard it as manifesting a strength of the ABM methodology: scholars from across the disciplines, who may not share literature, constructs, or methodologies, can nevertheless critique each other’s work pointedly—ABM is an interdisciplinary lingua franca (Abrahamson & Wilensky, 2005a). In fact, readers are welcome to download the model file and modify or replace the procedures so as to express their own hypotheses. Rationale of the Stratified Learning Zone model: Selection of Key Parameters, Hypothesizing Behavior Rules, and Authoring the Rules Within the NetLogo Environment Any model, regardless of the medium in which it is expressed, e.g., text, diagram, or agent-based model, is per force an attenuation of the “objective” reality. Initially, the modeler must use circumspection in answering the question, “What is the nature of the phenomenon we are attempting to model?” For example, we asked ourselves whether we are modeling: (a) a specific activity, i.e., “students collaborating on constructing the sample space of the binomial stochastic generator that has 9 variables each with the values “green” and “blue” that are glued onto purple construction paper”; or (b) “students collaborating on a task that demands a variety of roles that range by the content knowledge they foster.” We chose the latter option. Next, in building an agent-based model, one defines the agents (e.g., students, teacher) and any other objects at play (e.g., portable artifacts), and assigns the agents properties evaluated as relevant to the phenomenon under investigation, including constants (e.g., gender) and variables (e.g., role in collaborative activity). The modeler’s selection of these agents and properties is informed by a general rationale of the model, which the modeler articulates, e.g.: • Classroom objectives are mandated by a curriculum • A total of n individual students cluster in m groups of variable size; whom they group with is a mixture of student and teacher choice (teachers may opt to create either homogeneous or heterogeneous groups) • Individuals are reinforced by their group-mates for contributing toward a group’s objective, where ‘contribution’ is measured vis-à-vis the project specifications For the stratified learning zone (SLZ) model, we chose a puzzle task (see Figure 1, below). This linear puzzle consists of set of pieces that need to be concatenated according to a logical sequence. Necessary activities within this task are retrieving pieces (simplest task), connecting pieces (most demanding task), and verifying (intermediate demand). Thus, the roles that students might specialize in are piece-retrievers, piece-connectors, and puzzle-verifiers. Puzzle pieces are scattered all over the classroom. Retrievers wander around and, when they find a piece that they evaluate as useful (it may in fact be incorrect), they go back to their group’s table, deliver the piece to the connector and then return to retrieve more puzzle pieces. Upon receiving a piece, the piece-connector evaluates its fit to the puzzle in its current state. If the piece is not suitable, the piece-connector orders the piece-retriever to drop the piece somewhere else and bring a new one. If the piece is suitable, the piece-connector takes it and tries to add it to the puzzle. Once the puzzle is completed, the puzzle-verifiers check it. If one piece is out of place, the group has to re-assemble parts of the puzzle. For each task, students increase their skill (faster and/or more accurate). Overall group performance is evaluated by the correctness of the puzzles and time-to-completion. Our independent variables are: (a) pedagogical style (with or without mandated role rotation); (b) students’ initial skill level for each task and distribution of skill levels within student; and (c) task difficulty. Note: As a measure of achieving initial “reliability”—evaluating whether the model rationale indeed expresses what it purports to express—the modelers first worked individually and only then shared notes. We could thus partially validate our conjectures through inter-modeler triangulation. Abrahamson, Blikstein, & Wilensky (2007) [email protected] Figure 1. Design rationale for the Stratified Learning Zone agent-based model. Once the model rationale has been articulated, as above, the modeler couches the agents’ rules of interacting with each other and the environment in IF–THEN couplets and packages each topical set of rules in a procedure. These procedures express the researcher’s conceptual model. For example, the following procedure (simplified for rhetorical clarity) is for the retriever–connector interaction, and delineates the agent’s commands— retrievers gather pieces, connectors receive and evaluate pieces, and retrievers drop unfitting pieces.